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n  any  military  campaign,  commanders,  politicians,  and  the  general  public  all  de¬ 
sire  to  know  whether  the  effort  is  succeeding.  For  conventional  conflicts,  well 
developed  theories  of  war  give  a  good  understanding  of  the  objectives  to  pursue 
and  how  to  pursue  them.  These  theories  also  enable  the  derivation  of  well  de¬ 
fined  metrics  for  progress,  such  as  terrain  held,  numbers  of  enemy  fighters  killed 
or  captured,  or  amount  of  enemy  equipment  and  materiel  destroyed.  In  uncon¬ 
ventional  conflicts  the  theories  of  war  are  more  complex,  objectives  and  ways  to 
achieve  them  are  less  straightforward,  and  notions  of  “winning”  and  “losing”  are 
more  difficult  to  define.  As  a  result,  it  is  also  more  difficult  to  gauge  and  demon¬ 
strate  progress  in  such  conflicts.  For  the  specific  case  of  counterinsurgency,  how¬ 
ever,  gauging  and  demonstrating  progress  is  at  least  as  important  as  in  a  conven¬ 
tional  war,  since  the  former  tends  to  last  longer  and  therefore  requires  sustained 
political  and  public  support  to  conduct — and  such  support  is  often  tied  to  proof 
of  progress.  Thus  operations  assessment,  designed  to  show  whether  progress  is 
being  made,  should  be  a  vital  part  of  any  unconventional  conflict,  especially 
counterinsurgency. 

For  the  current  conflict  in  Afghanistan,  assessments  of  progress  have  been 
highly  criticized.  Early  in  the  war,  efforts  to  measure  and  demonstrate  progress 
were  relatively  immature,  as  evidenced  by  the  “initial  assessment”  prepared  by 

General  Stanley  McChrystal  soon  after  he  took  com¬ 
mand  of  the  International  Security  Assistance  Force 
(ISAF)  in  2009: 
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ISAF  must  develop  effective  assessment  architectures  . . . 
to  measure  the  effects  of  the  strategy,  assess  progress  to¬ 
ward  key  objectives,  and  make  necessary  adjustments. 
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ISAF  must  identify  and  refine  appropriate  indicators  to  assess  progress,  clarifying  the 

difference  between  operational  measures  of  effectiveness  critical  to  practitioners  on 

the  ground  and  strategic  measures  more  appropriate  to  national  capitals.1 

The  fact  that  General  McChrystal  did  not  find  such  processes  and  products  in 
place  when  he  took  command  in  2009  implies  that  we  were  poorly  assessing 
progress  eight  years  into  the  war.  In  part  because  there  was  no  single  effective 
campaign  assessment  at  that  time,  many  groups  then  took  it  upon  themselves  to 
create  one.  At  a  recent  NATO  conference  an  attendee-generated  list  showed  over 
twenty  different  campaign-level  operations  assessments  being  prepared  by  vari¬ 
ous  organizations  for  Afghanistan.2  Clearly,  the  importance  of  assessing  prog¬ 
ress  in  the  campaign  has  been  realized,  and  considerable  effort  has  been  exerted 
to  improve  our  assessment  capabilities.  However,  criticisms  of  our  ability  to 
measure  and  demonstrate  progress  in  a  clear,  credible,  and  transparent  manner 
have  only  increased.  The  reasons  for  these  criticisms  vary  considerably,  from 
confusion  in  planning  to  shortfalls  in  availability  of  data  or  in  doctrine,  to  flaws 
of  current  processes  and  products.3  By  far  the  most  popular  criticism,  however, 
is  that  we  do  not  have  the  right  metrics  for  Afghanistan.  Many  papers  have  been 
published  on  this  subject,  and  in  2010  I  was  invited  to  no  fewer  than  three  con¬ 
ferences,  each  convened  to  generate  a  better  list  of  metrics  for  Afghanistan.4 
Even  the  former  head  of  the  ISAF  Afghan  Assessments  Group  (AAG)  is  on  rec¬ 
ord  as  saying,  “Our  metrics  suck.”5 

Given  these  efforts,  it  seems  clear  that  what  should  be  a  vital  part  of  the  cam¬ 
paign  in  Afghanistan  is  not  going  well.  But  if  the  problem  were  simply  one  of 
finding  the  right  metrics,  it  seems  likely  the  solution  would  have  been  found  by 
now,  especially  since  similar  criticisms  were  levied  during  the  war  in  Iraq.5  Based 
on  my  five  years  of  personal  experience  with  operations  assessments  in  Iraq,  in 
Afghanistan,  and  at  several  commands  (e.g.,  U.S.  Central  Command),  I  submit 
that  the  problem  goes  beyond  the  wrong  metrics  and  that  more  fundamental 
problems  with  operations  assessment  exist — for  Afghanistan,  for  Iraq,  and  in 
general.  As  I  will  show,  operations  assessments  suffer  from  a  number  of  serious 
issues  that  feed  upon  and  reinforce  each  other.  The  resulting  “failure  cycle”  is  the 
reason  why  the  theoretical  utility  of  operations  assessment  is  rarely  realized  in 
practice,  and  for  the  specific  case  of  Afghanistan  it  is  a  large  contributor  to  our 
inability  to  measure  or  demonstrate  progress. 

DEFINITION,  PURPOSE,  AND  THEORETICAL  UTILITY  OF 
OPERATIONS  ASSESSMENT 

Before  launching  into  a  critique,  it  is  worth  reviewing  the  doctrinal  definition,  the 
purpose,  and  the  theoretical  utility  of  operations  assessment.  In  terms  of  defini¬ 
tion,  Joint  Publications  (JPs)  3-0  (Joint  Operations)  and  5-0  (Joint  Operation 
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Planning)  define  operations  assessment  as  “a  process  that  measures  progress  of 
the  joint  force  toward  mission  accomplishment”;7  the  U.S.  Army’s  Field  Manual 
(FM)  5-0,  Operations  Process,  says  it  is  “the  continuous  monitoring  and  evaluation 
of  the  current  situation,  particularly  the  enemy,  and  progress  of  an  operation.”8 
The  Army’s  counterinsurgency  manual  (FM  3-24)  defines  operations  assessment 
as  “the  continuous  monitoring  and  evaluation  of  the  current  situation  and  prog¬ 
ress  of  an  operation.”9 

These  definitions  highlight  the  main  purpose  in  conducting  operations  as¬ 
sessment,  which  is  to  measure  the  progress  of  an  operation  toward  accomplish¬ 
ing  its  mission.  But  these  documents  elaborate  in  a  variety  of  ways: 

•  [JPs  3-0  and  5-0]  Commanders  adjust  operations  based  on  their  assessments  to 
ensure  that  military  objectives  are  met  and  the  military  end  state  is  achieved. 

The  assessment  process  is  continuous  and  directly  tied  to  the  commander’s  deci¬ 
sions.  Assessment  actions  and  measures  help  commanders  adjust  operations  and 
resources  as  required,  determine  when  to  execute  “branches  and  sequels”  [op¬ 
tional  or  successive  operations  envisioned  in  a  plan  of  action],  and  make  other 
critical  decisions  to  ensure  current  and  future  operations  remain  aligned  with 
the  mission  and  military  end  state.10 

•  [FM  5-0]  Assessment  involves  deliberately  comparing  forecasted  outcomes  with 
actual  events  to  determine  the  overall  effectiveness  of  force  employment.  Assess¬ 
ment  helps  the  commander  determine  progress  toward  attaining  the  desired  end 
state,  achieving  objectives,  and  performing  tasks.  It  also  involves  continuously 
monitoring  and  evaluating  the  operational  environment  to  determine  what 
changes  might  affect  the  conduct  of  operations." 

•  [FM  3-24]  Effective  assessment  is  necessary  for  commanders  to  recognize 
changing  conditions  and  determine  their  meaning.  It  is  crucial  to  successful 
campaign  adaptation  and  innovation  by  commanders.  Assessment  is  a  learning 
activity  and  a  critical  aspect  of  [campaign]  design.  This  learning  leads  to  rede¬ 
sign.  Therefore,  [campaign]  design  can  be  viewed  as  a  perpetual  design-learn 
[assess]-redesign  activity.12 

From  these  and  other  documents  on  the  subject  one  can  compile  a  set  of 
points  describing  possible  theoretical  purposes  for,  and  utilities  of,  operations 
assessment.  These  include  informing  commanders’  decision  making  (e.g.,  on 
resource  allocation);  completing  the  planning  or  design  cycle  (i.e.,  “observe- 
plan-execute-assess,”  or  “design-learn-redesign”);  recognizing  changing  condi¬ 
tions  in  the  environment;  stimulating  and  informing  adaptation  and  innova¬ 
tion;  reducing  uncertainty  and  bounding  risk;  showing  causal  linkages  between 
actions  and  the  achievement  of  objectives;  documenting  the  commander’s 
decision-making  process;  and  evaluating  performance  of  subordinate  units. 
While  these  items  are  not  all-inclusive,  the  list  certainly  comprises  actions  that 
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most  commanders  would  find  useful.  The  question  is:  How  well  do  operations 
assessments  perform  these  functions  in  practice?  In  my  experience  with  assess¬ 
ments  for  Iraq  and  Afghanistan,  and  in  the  experience  of  several  objective  ob¬ 
servers  for  Afghanistan,  the  answer  is  not  very  well . 13 

REASONS  OPERATIONS  ASSESSMENTS  FAIL 

There  are  many  reasons  why  operations  assessments  fail,  by  which  I  mean  that  in 
practice  they  do  not  realize  the  theoretical  utilities  listed  above.  I  will  focus  on  a 
few  key  reasons,  chosen  because  they  are  particularly  important  and  because 
they  result  in  a  cascading  chain  of  issues  that  reduce  the  effectiveness  of  opera¬ 
tions  assessments  and  ensure  the  propagation  of  these  issues  into  the  future. 

Doctrinal  Deficiencies 

One  reason  operations  assessments  fall  short  is  that  there  are  deficiencies,  con¬ 
tradictions,  and  confusion  in  the  doctrine  that  is  supposed  to  guide  their  con¬ 
duct.  For  joint  military  operations,  the  first  stop  for  doctrine  is  the  joint 
publications,  especially  JPs  3-0  and  5-0.  Unfortunately,  when  it  comes  to  opera¬ 
tions  assessment,  these  publications  are  notably  vague.  While  they  do  offer  guid¬ 
ance  on  the  purpose  of  conducting  assessments,  they  mainly  focus  on  making 
clear  the  distinctions  between  “measures  of  effectiveness”  (MoEs)  and  “mea¬ 
sures  of  performance”  (MoPs).  Nowhere  do  they  discuss  in  detail  how  to  do  op¬ 
erations  assessment.  Thus,  to  a  practitioner  they  provide  little  more  than  a 
beginner’s  lesson  in  vocabulary. 

The  doctrine  issued  by  individual  services  is  another  source  of  guidance. 
Field  Manual  5-0  provides  more  detail  on  how  to  do  assessment,  and  much  of  its 
guidance  is  useful  and  sound  from  a  practitioner’s  viewpoint.  However,  its  guid¬ 
ance  also  contains  many  contradictions  that  detract  from  its  overall  utility.  Ex¬ 
amples  include: 

•  FM  5-0  says  detailed  analysis  is  to  be  avoided,  that  “committing  valuable 
time  and  energy  to  developing  excessive  and  time-consuming  assessment 
schemes  squanders  resources  better  devoted  to  other  operations  process  ac¬ 
tivities.”14  However,  it  says  later  that  “establishing  cause  and  effect  is  some¬ 
times  difficult,  but  crucial  to  effective  assessment.  Commanders  and  staffs 
are  well  advised  to  devote  the  time,  effort,  and  energy  needed  to  properly 
uncover  connections  between  causes  and  effects.”15  While  the  latter  may 
seem  straightforward,  in  practice  it  is  typically  very  time-consuming. 

•  It  stresses  the  need  to  incorporate  quantitative  and  qualitative  indicators  in 
the  assessment,  observing  that  “the  appropriate  balance  depends  on  the 
situation — particularly  the  nature  of  the  operation  and  available  resources 
for  assessment — but  rarely  lies  at  the  ends  of  the  scale.”1”  However,  in  the 
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FIGURE  1 

SAMPLE  ASSESSMENT  FRAMEWORK 


FM  5-0,  app.  H. 


manual’s  appendix  H,  which  explains  how  to  develop  a  formal  assessment 
plan,  the  sole  assessment  framework  presented  is  strictly  quantitative 
(figure  1). 

•  It  describes  the  framework  in  figure  1  as  “a  hierarchy  used  in  formal  assess¬ 
ments  that  numerically  describes  progress  toward  achieving  desired  condi¬ 
tions.”  It  recommends  such  an  assessment  be  combined  with  “expert 
opinions  of  members  of  the  staff,  subordinate  commanders,  and  other 
partners.  In  this  way,  the  commander  receives  both  a  mathematically  rigor¬ 
ous  analysis  as  well  as  expert  opinions.”17  While  this  may  seem  true  on  its 
face,  the  sample  framework  is  actually  not  mathematically  (or  even  logi¬ 
cally)  rigorous.  It  involves  weight-averaging  numbers  with  different  units, 
thereby  comparing  “apples  and  oranges.”  Also,  the  weights  used  in  the 
framework  are  entirely  subjective  (and  likely  arbitrary),  thereby  undermin¬ 
ing  its  “mathematical  rigor.”  Finally,  the  framework  implies  a  model  of  war¬ 
fare  in  which  all  actions  are  independent  and  so  can  (along  with  their 
effects)  be  counted,  added,  and  averaged  together.  This  is  highly  unlikely  to 
be  true  in  a  military  campaign. 
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These  contradictions  regarding  the  importance  of  analysis  and  intellectual  rigor 
and  the  balance  between  quantitative  and  qualitative  information  are  just  a  few 
of  the  confusing  aspects  of  Field  Manual  5-0  that  reduce  its  usefulness  to  practi¬ 
tioners  of  operations  assessment. 

For  counterinsurgency,  FM  3-24  has  only  three  pages  on  how  to  conduct  as¬ 
sessments.  While  one  of  them  gives  a  useful  set  of  example  indicators,  nowhere 
does  the  manual  discuss  how  to  structure  an  assessment  framework  or  product, 
how  to  collect  and  analyze  data,  etc. — which  is  odd,  given  that  it  stresses  the  crit¬ 
ical  role  that  assessments  play  in  design,  adaptation,  and  redesign.  Overall,  it 
provides  little  in  the  way  of  value  to  practitioners  of  operations  assessment. 

Some  might  reply  that  doctrine  exists  to  provide  broad  guidance  and  that  we 
should  not  expect  it  to  provide  detailed  instructions  on  how  to  conduct  opera¬ 
tions  assessment.  Instead,  some  suggest,  practitioners  should  look  to  “best  prac¬ 
tices”  guides,  such  as  one  produced  by  the  Center  for  Army  Lessons  Learned.11’ 
However,  even  that  handbook  has  its  deficiencies:  it  simply  rehashes  much  of 
FM  5-0,  its  descriptions  of  the  roles  of  military  echelons  are  unrealistic,  it  con¬ 
tains  few  helpful  examples  of  assessment  products,  and  it  argues  throughout 
that  the  U.S.  Agency  for  International  Development’s  Tactical  Conflict  Assess¬ 
ment  and  Planning  Framework  model  should  be  the  foundation  of  stability  op¬ 
erations  assessment  without  justifying  why  or  explaining  how  a  tactical  model 
could  be  used  to  assess  progress  at  the  operational  level.  Thus  even  our  assess¬ 
ment  handbooks  provide  little  value  to  practitioners.19 

In  addition,  there  is  confusion  in  our  doctrine  as  to  whether  the  principles  of 
“effects-based  operations”  (EBO)  still  apply.  General  James  N.  Mattis,  then 
Commander,  U.S.  Joint  Forces  Command,  instructed  his  organization  in  2008 
that  the  terms  and  concepts  associated  with  EBO  were  to  be  stricken  from  joint 
doctrine,  training,  and  education. 20  Yet  it  remains  unclear  whether  EBO  should 
continue  to  be  used,  and  one  study  of  Afghanistan  concluded  that  “EBO  and 
EBA  [effects-based  assessment]  are  alive  and  well.”21  This  is  true  in  my  experi¬ 
ence  also,  and  it  is  perhaps  not  surprising,  since  Joint  Publications  3-0  and  5-0 
have  not  been  fully  updated  since  General  Mattis’s  memorandum.  While  the 
Army’s  FM  5-0  was  published  afterward,  it  still  contains  references  to  effects, 
and  its  sample  assessment  framework  simply  replaces  “desired  effects”  with  “de¬ 
sired  conditions”  (figure  l).22  Thus  even  our  planning  doctrine  is  confusing  and 
deficient.  In  any  case,  even  if  planners  do  not  use  EBO,  practitioners  of  opera¬ 
tions  assessment  often  still  use  an  effects-based  approach,  because  it  is  all  they 
can  find  in  doctrine  (figure  1).  In  these  cases,  it  is  prudent  to  ask  whether  one 
should  expect  an  effects-based  assessment  to  succeed,  since  these  efforts  amount 
to  “cherry-picking”  aspects  of  coherent  doctrinal  processes.  Indeed,  I  have  yet  to 
see  this  approach  succeed  in  practice. 


SCHRODEN 


95 


Lack  of  Training  for  Practitioners 

Another  reason  operations  assessments  fail  is  that  those  who  produce  them  are 
not  adequately  trained.  In  my  experience,  two  types  of  people  get  tasked  to  con¬ 
duct  operations  assessment:  staff  officers  who,  regardless  of  their  skill  sets,  have 
been  placed  in  assessments  billets  (many  of  them  former  pilots,  for  some  rea¬ 
son);  and  “ORSAs”  (individuals  formally  trained  in  operations  research  and  sys¬ 
tems  analysis).  Neither  of  these  groups  receives  any  specific  training  on  how  to 
conduct  operations  assessment — they  are  typically  left  to  decipher  doctrine  on 
their  own  or  to  hunt  for  assessment  products  created  by  others  that  they  can 

23 

copy. 

Given  the  deficiencies  and  confusion  that  exist  in  doctrine,  it  is  not  surprising 
that  many  practitioners  fail  in  their  attempts  to  devise  useful  assessment  processes 
from  scratch.  Those  who  find  fully  formed  assessment  products  from  another 
command  will  usually  fail,  because  the  products  they  copy  are  typically  deriva¬ 
tions  of  the  framework  in  figure  1,  which  suffers  from  the  drawbacks  identified 
earlier.  Anyone  who  has  attended  conferences  on  operations  assessment  can  at¬ 
test  that  the  approaches  presented  tend  to  use  that  structure,  with  the  same 
weighted-average  “roll-ups”  of  metrics  into  the  same  “stoplight  chart”  (i.e., 
red/yellow/green  coding)  products.  In  the  absence  of  sound  doctrine  and  train¬ 
ing,  we  have  left  practitioners  either  to  flounder  on  their  own  or  to  steal  flawed 
products  from  others,  both  of  which  are  recipes  for  failure. 

Expectations  of  Audiences 

Operations  assessments  also  fail  because  in  practice  they  rarely  live  up  to  the  ex¬ 
pectations  of  commanders.  More  specifically,  though  commanders  establish  as¬ 
sessment  cells  because  they  desire  to  reap  the  theoretical  benefits  of  operations 
assessment  identified  earlier,  practitioners  of  assessment  are  set  up  for  failure  by 
doctrinal  and  training  shortcomings;  the  results  tend  to  be  processes  and  prod¬ 
ucts  that  do  not  deliver  the  theoretical  utilities  that  commanders  expect.  When 
commanders  realize  this,  they  stop  paying  attention  to  assessments,  which  leads 
to  the  slow  death  of  the  latter.  A  related  issue  is  that  commanders  who  do  not  see 
the  theoretical  promise  of  operations  assessment  translated  into  practical  utility 
do  not  go  on  to  be  advocates  for  the  process.  This  indifference  to  assessment  al¬ 
lows  poor  doctrine  and  practices  to  persist,  since  if  commanders  lose  interest  in 
assessment  while  in  command,  they  certainly  will  not  care  about  assessment 
thereafter. 

BREAKING  THE  FAILURE  CYCLE 

If  we  now  look  more  broadly  at  the  reasons  operations  assessments  fail,  it  is  clear 
they  are  not  isolated  and  independent;  rather,  they  are  linked  together  in  what  I 
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call  a  “failure  cycle.”  This  cycle,  which  is  de¬ 
picted  in  figure  2,  runs  as  follows:  poor  and 
confusing  doctrine  leads  (in  part)  to  inade¬ 
quate  (or  no)  training  of  assessment  practi¬ 
tioners,  which  leads  to  poor  assessment 
processes  and  products,  which  leads  to  com¬ 
manders  who  are  uninterested  in  assessment, 
which  leads  to  a  lack  of  advocacy  for  fixing 
assessment,  which  leads  to  a  perpetuation  of 
poor  doctrine — and  the  cycle  continues. 

How  can  the  failure  cycle  for  operations 
assessment  best  be  broken?  In  principle,  one 
could  start  at  any  point  in  the  cycle,  but  in 
practice  certain  spots  would  be  easier  or 
more  logical  than  others. 

Gaining  an  Advocate.  While  on  their  face 
other  aspects  of  the  failure  cycle  may  seem  more  important,  the  lack  of  advocacy 
within  the  Department  of  Defense  (and  other  departments)  for  operations  as¬ 
sessment  is  in  fact  the  most  crucial  problem,  for  several  reasons.  First,  without 
an  advocate  to  highlight  to  the  department  that  its  current  doctrine  and  pro¬ 
cesses  are  inadequate,  there  will  be  no  impetus  for  change.  It  is  too  easy  to  repub¬ 
lish  or  slightly  tweak  current  doctrine  rather  than  to  rethink  completely  the  way 
in  which  operations  assessments  are  designed  and  implemented — and  in  the 
quasi-post-EBO  environment  in  which  plans  are  currently  being  written  and  as¬ 
sessed,  a  complete  rethinking  is  required.  Second,  without  an  advocate  there  is 
no  center  of  gravity  around  which  to  accumulate  knowledge  and  thus  there  will 
never  be  an  established  cadre  of  experts  in  operations  assessment.  Instead,  we 
will  continue  to  cannibalize  other  military  occupational  specialties,  most  nota¬ 
bly  the  ORSA  pool,  to  conduct  assessments.  Thus,  the  first  step  in  breaking  the 
failure  cycle  is  to  gain  a  high-level  advocate  for  operations  assessment  within  the 
Defense  Department. 

Improving  Doctrine.  Once  an  advocate  is  gained  and  the  argument  can  be  made 
at  the  right  levels  that  our  doctrine  needs  to  be  dramatically  improved,  the  first 
issue  to  be  addressed  will  be  whether  “effects-based  operations”  is  still  an  opera¬ 
tive  planning  process  for  the  U.S.  military.  This  is  a  larger  issue  than  assessment, 
but  the  way  in  which  planning  is  conducted  directly  impacts  the  way  in  which 
assessments  are  conducted.  If  the  U.S.  military  decides  to  keep  EBO,  perhaps 
simply  adding  more  detail  to  Joint  Publications  3-0  and  5-0  (and  Field  Manual 
3-24)  and  fixing  the  contradictions  in  FM  5-0  will  suffice.  What  seems  more 
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likely  is  that  some  new  planning  construct  would  come  to  the  fore  in  such  a  dis¬ 
cussion  and  that  as  a  result  we  would  need  to  revisit  fundamentally  the  purpose 
and  design  of  operations  assessment.  Regardless,  those  who  are  responsible  for 
improving  this  doctrine  need  to  understand  that  assessing  the  progress  of  mili¬ 
tary  operations  (especially  counterinsurgencies)  is  difficult  and  that  therefore 
doctrine  needs  to  provide  much  more  detail  on  how  to  do  it. 

Improving  Training.  Improving  doctrine,  though  necessary,  will  not  in  itself  im¬ 
prove  the  ability  of  practitioners  to  conduct  assessments,  for  two  reasons.  First, 
there  is  currently  no  “training  pipeline”  for  practitioners  of  assessment,  so  even 
when  doctrine  is  fixed,  practitioners  in  the  field  will  still  flounder.  Second,  there 
is  no  dedicated  cadre  of  experts  in  the  practice  of  operations  assessment,  so  even 
if  a  training  program  is  designed,  those  who  go  through  it  will  inevitably  revert 
to  their  primary  military  occupational  specialties  and  their  knowledge  and  expe¬ 
rience  will  be  lost.  This  second  reason  creates  two  further  issues:  there  will  be  a 
lack  of  feedback  from  the  field  to  the  schoolhouse,  so  the  development  of  better 
assessment  techniques  will  stagnate;  and  there  will  be  no  pipeline  of  advocates  for 
assessment  to  replace  the  initial  advocate  called  for  above.  Thus,  improving  train¬ 
ing  for  operations  assessment  relies  on  three  factors:  improvement  of  doctrine,  a 
formal  course  of  instruction,  and  establishment  by  the  personnel-management 
community  of  a  military  occupational  specialty  for  operations  assessment. 

The  latter  deserves  further  elaboration.  There  is  a  popular  belief  that  ORSAs 
are  trained  to  conduct  operations  assessment,  when  in  fact  they  are  not.  I  believe 
this  stems  from  a  broader  confusion  of  the  terms  “operations  assessment”  and 
“analysis”  (or  “operations  research”).  A  practical  way  of  differentiating  the  two 
might  be  to  say  that  operations  assessment  focuses  on  measuring  the  progress  of 
an  operation,  while  operations  analysis  focuses  on  optimizing  the  performance 
of  units  and  individuals  (i.e.,  the  organization)  conducting  the  operation.  These 
are  distinct  activities.  Accordingly,  we  should  stop  presuming  that  people 
trained  in  operations  analysis  are  somehow  experts  in  operations  assessment. 
Additionally,  we  should  realize  that  by  tasking  ORSAs  with  operations  assess¬ 
ment  we  are  unconsciously  sacrificing  our  capability  to  conduct  operations 
analysis  (i.e.,  to  optimize  our  performance).  Hence  my  assertion  that  what  is  re¬ 
quired  is  both  a  formal  course  of  instruction  for  operations  assessment  and  a 
dedicated  military  occupational  specialty  in  it. 

Improving  Processes  and  Products.  It  would  be  easier  to  design  better  assessment 
processes  and  products  were  doctrinal  and  training  issues  resolved,  but  in  the 
interim  there  are  steps  that  can  be  taken.  To  begin,  practitioners  of  operations 
assessment  should  abandon  the  sample  framework  in  FM  5-0  (figure  1).  As  dis¬ 
cussed  above,  it  does  not  balance  qualitative  and  quantitative  information  as 


98 


NAVAL  WAR  COLLEGE  REVIEW 


doctrine  (and  common  sense)  dictates,  nor  is  it  “mathematically  rigorous.”  Ade¬ 
quate  arguments  in  support  of  this  notion  have  been  made,  and  I  will  not  rehash 
them  here.24  Instead,  I  will  highlight  three  key  shifts  in  thinking  that,  if  imple¬ 
mented  in  the  field,  would  go  a  long  way  toward  improving  our  current  attempts 
at  assessment. 

First,  it  is  absolutely  necessary  to  balance  quantitative  and  qualitative  informa¬ 
tion.  While  it  is  easier  to  work  with  numbers  and  their  extensive  use  tends  to  en¬ 
hance  the  appearance  of  objectivity  and  robustness  of  assessment  (if  only  through 
a  facade  of  rigor),  from  a  practical  viewpoint  it  is  silly  to  expect  that  one  can  mea¬ 
sure  the  progress  of  a  military  operation  through  quantitative  means  alone.  Thus, 
instead  of  eschewing  qualitative  information  as  “unreliable”  or  “too  subjective,” 
we  should  embrace  both  qualitative  and  quantitative  sources,  so  long  as  the  infor¬ 
mation  is  useful  in  addressing  progress  toward  mission  accomplishment. 

The  second  shift  in  thinking  is  to  move  away  from  slide  shows  and  stoplight 
charts  as  the  products  of  operations  assessment.  A  recommendation  made  else¬ 
where  that  practitioners  move  toward  narrative  formats  for  their  products  is 
solid  and  should  be  accepted.2"  Again,  from  a  practical  viewpoint,  it  is  naive  to 
think  that  something  as  simple  as  a  colored  map  or  a  series  of  red,  yellow,  and 
green  circles  can  convincingly  communicate  progress  in  something  as  complex 
as  warfare.  Such  presentations  inevitably  engender  questions  from  the  audience 
that  require  further  explanation;  arguments  that  they  can  stand  on  their  own  are 
contrary  to  empirical  observation.  While  narratives  can  be  more  time-consuming 
both  to  write  and  to  read,  for  assessment  they  have  a  number  of  advantages:  they 
allow  variations  and  nuances  across  the  area  of  operations  to  be  captured  and 
appreciated;  they  remind  people  of  the  context  and  complexity  of  the  operation; 
they  force  assessors  to  think  through  issues  and  ensure  that  their  assessment  is 
based  on  rigorous  thought;  and  they  are  the  only  way  to  ensure  that  a  proper  bal¬ 
ance  is  struck  between  quantitative  and  qualitative  information,  analysis  and 
judgment,  and  empirical  and  anecdotal  evidence.26 

The  third  shift  in  thinking  is  to  realize  that  to  assess  progress  in  a  modern  mil¬ 
itary  operation  properly,  it  is  necessary  to  gather,  analyze,  and  fuse  information 
on  the  activities  of  enemy  (“red”),  civilian  (“white”),  and  friendly  (“blue”) 
forces.27  Our  military  is  not  well  suited  to  doing  this.  Currently,  intelligence  or¬ 
ganizations  focus  on  information  pertaining  to  the  enemy  and  to  a  lesser  extent 
on  civilian  activities,  and  the  good  ones  perform  some  fusion  of  the  two.28  As 
highlighted  above,  operations  analysts  typically  gather  and  analyze  information 
about  blue  forces.  However,  there  is  no  entity  that  currently  specializes  in  fusing 
and  analyzing  information  across  the  red,  white,  and  blue  spectrum.  This  is  an 
area  in  which  future  operations  assessment  cells  could  look  to  specialize. 
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Currently,  though,  what  one  finds  in  a  place  like  Afghanistan  is  a  cadre  of 
people  gathering  and  analyzing  information  on  the  enemy,  a  much  smaller 
group  focused  on  civilians,  and  hardly  anyone  gathering  and  analyzing  material 
on  blue-force  activities  (largely  because  the  bulk  of  ORSAs  are  manning  assess¬ 
ments  cells).  This  absence  of  blue-force  data  collection  and  analysis  and  of 
red/ white/blue  fusion  severely  constrains  our  ability  to  link  blue-force  actions 
with  changes  observed  in  the  environment.  These  are  serious  problems  that  have 
not  yet  been  widely  appreciated  as  primary  reasons  why  assessments  for  Afghan¬ 
istan  are  failing.  Recognition  would  go  far  toward  improving  current  assess¬ 
ments  and  would  induce  a  further  realization  that  more  emphasis  needs  to  be 
placed  on  data  collection,  management,  and  analysis  resources  across  the  red/ 
white/blue  spectrum  for  future  military  operations. 

Increasing  the  Interest  of  Commanders.  If  the  above  issues  were  worked  out,  the 
indifference  of  commanders  as  part  of  the  failure  cycle  would  likely  fix  itself, 
since  many  of  the  theoretical  promises  of  assessment  would  be  realized  in  fact. 
However,  two  additional  steps  could  be  taken  to  ensure  that  once  commanders 
become  interested  in  assessments  they  stay  interested.29  The  first  is  to  include  in 
the  training  that  commanders  receive  at  least  a  cursory  discussion  of  operations 
assessments,  their  purpose  and  utility  and  how  they  can  be  effective  tools  for 
measuring  and  communicating  progress  (this  would  be  useful  for  planners  to 
hear  as  well).  This  discussion  should  be  facilitated  by  someone  with  experience 
with  assessments  in  the  field.  Second,  commanders  should  be  instructed  as  to 
the  importance  of  their  own  involvement  in  the  assessment  process,  since  if  a 
commander  does  not  back  the  process  the  staff  will  quickly  stop  supporting  it. 
Additionally,  commanders  can  use  the  process  as  a  means  of  articulating  and  pe¬ 
riodically  adjusting  their  guidance  to  the  staff  regarding  broader  intent,  priori¬ 
ties,  and  focus  of  effort. 

TIME  FOR  A  RESET 

The  problems  with  operations  assessment  run  much  deeper  than  simply  having 
poor  metrics.  There  is  an  entire  failure  cycle  at  work,  and  until  its  associated  is¬ 
sues  are  rectified  the  theoretical  promises  of  operations  assessment  will  con¬ 
tinue  to  go  unrealized.  To  recap,  these  issues  are  identifying  an  advocate  for 
assessments,  fixing  our  planning  and  assessment  doctrine,  creating  a  military 
occupational  specialty  and  formal  course  of  instruction  for  operations  assess¬ 
ment,  and  shifting  our  thinking  away  from  strictly  quantitative  and  picture- 
based  assessment  products  toward  balanced,  comprehensive,  analytic  narratives. 
Until  and  unless  these  issues  are  addressed,  my  overarching  recommendation  is  to 
stop  doing  operations  assessments  altogether.  The  bulk  of  current  assessment 
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products  and  processes  for  Afghanistan,  for  example,  do  as  much  harm  as  good. 
As  has  been  argued,  they  consistently  undermine  the  transparency  and  credibil¬ 
ity  of  military  judgment,  because  they  themselves  are  neither  transparent  nor 
credible.30  Additionally,  current  efforts  on  generating  better  metrics  are  simply 
tweaking  the  margins  of  a  much  larger  problem.  Until  the  failure  cycle  is  com¬ 
pletely  and  comprehensively  fixed,  we  should  stop  pretending  that  assessments 
are  playing  a  useful  role  and  acknowledge  the  opportunity  cost  of  using  un¬ 
trained  staff  officers  and  specialists  in  operations  research  and  systems  analysis 
to  conduct  them.  Overall,  we  would  be  better  served  to  take  a  “time-out”  on  as¬ 
sessments,  fix  the  failure  cycle,  and  come  back  with  an  improved  approach.  Con¬ 
tinuing  on  our  current  circular  path  will  simply  ensure  that  progress  in  the  next 
war  will  be  as  difficult  to  measure  as  progress  in  our  current  wars. 
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